Hybrid Autoregressive and Non-Autoregressive Transformer Models for Speech Recognition

The autoregressive (AR) models, such as attention-based encoder-decoder models and RNN-Transducer, have achieved great success in speech recognition. They predict the output sequence conditioned on previous tokens acoustic encoded states, which is inefficient GPUs. non-autoregressive (NAR) can get rid of temporal dependency between entire one inference step. However, NAR model still faces two major problems. Firstly, there a gap performance advanced AR models. Secondly, it’s difficult for most to train converge. We propose hybrid transformer (HANAT) model, integrates deeply by sharing parameters. assume that will assist learn some linguistic dependencies accelerate convergence. Furthermore, two-stage applied improve performance. All experiments are conducted mandarin dataset ASIEHLL-1 english librispeech-960 h. results show HANAT achieve competitive with outperform many complicated Besides, RTF only 1/5 model.